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Abstract 


An increasing emphasis on principals as key to school improvement has contributed to efforts to elevate 
principal effectiveness that have taken various forms across the US. The primacy of the state as the focal 
point of educational reform elevates the value of understanding commonalities and differences among states 
in characteristics of principals, the distribution of principals among schools and ultimately the policies 
associated with more effective school leadership, particularly for disadvantaged children. This paper 
describes major state policies, the distribution of elementary school principals among schools along a 
several dimensions, and pathways to the principalship to illustrate similarities and differences among six 
states in the tenure and experience distributions and how these vary by student demographic characteristics 
and district size. Measurement of principal effectiveness and its relationship with principal characteristics 
and state policies would be ideal, but complications introduced by the dynamics of principal influences and 
confounding effects of other factors inhibit this effort. Nonetheless, school value added to achievement 
provides information on differences in principal effectiveness, and we report within-school variation value 
added across principal regimes and the associations between value added and principal characteristics. The 
analysis reveals many similarities and some differences among the states, some of which are related to 
differences in governance structures. Perhaps the most striking differences relate to the pathways to the 


principalship including the fraction of principals with experiences as assistant principals and teachers. 


1. The Importance of School Leadership 

The introduction of school accountability and evidence on the substantial variation in 
teacher effectiveness have contributed to a growing emphasis on school leadership as central to 
school improvement efforts. Principals play a central role that includes teacher evaluation and 
mentoring, the creation of professional learning communities, as well as overseeing discipline 
and managing operations. 

The primacy of the state as the focal point of educational reform elevates the value of 
understanding commonalities and differences among states in characteristics of principals, the 
distribution of principals among schools and ultimately the policies associated with more 
effective school leadership, particularly for disadvantaged children. In this paper, we describe 
major state policies, the distribution of elementary and secondary school principals among 
schools along several dimensions, and pathways to the principalship to illustrate similarities and 
differences among states in the tenure and experience distributions and how these vary by 
student demographic characteristics and district size. The states include Georgia, Massachusetts, 
Missouri, North Carolina, Texas, and Washington. Note that the choice of states reflects data 
availability; the states are not nationally representative, but they are states in which the long 
administrative data panels allow detailed investigation of the pathways to the principalship. 
Similarly, we focus on schools that include grades 3-8 (i.e., we exclude secondary schools and 
small schools with non-standard grade configurations, such as K-2) based on the availability of 
annual achievement scores. 

Next we measure variation between principals in value-added to student achievement, 
and the predictive power of principal characteristics over value-added. Principals almost 


certainly contribute to achievement growth and the quality of instruction. However, this 


component of the analysis does not necessarily identify the causal contributions of principals to 
student test achievement—we elaborate on this caveat to these estimates below. 

State administrative longitudinal data on students and educators make this study possible. 
In particular, key to our analyses is the ability to follow students as they move through grades 
regardless of whether they remain in the same school attendance zone or switch schools or 
districts; this enables the measurement of achievement growth and estimation of school value 
added. An educator identifier enables similar tracking of educator careers. 

The descriptions of principal characteristics, pathways to the principalship, and within- 
school variation over time in student achievement associated with principal spells reveal many 
similarities and some differences among the states. Perhaps the most striking differences relate to 
the pathways to the principalship. In Georgia, North Carolina, and Texas more than 75 percent or 
principals had prior experience as assistant principals; by contrast, that number falls to 50 
percent or below for Massachusetts, Missouri, and Washington. Massachusetts serves as a real 
outlier along this dimension, as even in large districts with more than 10,000 students less than 
one third of the principals had assistant principal experience. 

Section 2 discusses the state administrative data and major policies and regulations 
related to school leadership, focusing on the commonalities and differences among the states. 
Section 3 describes the distributions of principal experience and tenure by student demographics 
and the distributions of principals by prior administrative and teaching experience. Our 
descriptions of the pathways to the principal position consider not only the shares with prior 
experience as an assistant principal and teacher but also the location of that experience to 
highlight variation among states in the hiring of principals with prior experience in the same 


district or even the same school. Section 4 examines variation in achievement growth within 


schools to highlight potential contributions of principals and systematic differences by 
experience and tenure. Finally, Section 5 summarizes the findings, discusses future work and 


considers implications for policy. 


23 Data and Principal Policies 

This section begins by describing the administrative data used by each state and presents 
basic descriptive statistics about enrollment and achievement. Next we outline major state 
policies that govern the school administrators. Given the similarities among the states we do not 
provide detailed descriptions for each. Rather we highlight salient differences along both the data 
and policy dimensions. 
2.a. Data 

The administrative data about principals, schools, and students that allow us to describe 
principal distributions and value added are provided by state departments of education.' The 
basic structure of the data is the same across all six states: principals and students are linked to 
schools and can be tracked over time even if they switch schools or, in the case of principals, 
roles and job titles. Although each state’s data are unique in the sense that they span different 
years and contain slightly different information, they have similar structures and contain much 


information in common. For instance, each state provides information on educator experience, 


' Specifically, from the GAA WARDS database in Georgia, which contains K-12 data from the Georgia Department 
of Education and is administered by the Governor’s Office of Student Achievement. Data for Massachusetts have 
been provided by the Department of Elementary and Secondary Education in Massachusetts. Data for Missouri have 
been provided by Missouri Department of Elementary and Secondary Education. The North Carolina data come 
from the North Carolina Education Research Data Center (NCERDC). This research was also made possible through 
data provided by the Texas Schools Project at the University of Texas at Dallas. Data for Washington have been 
provided by the Office of the Superintendent of Public Instruction in Washington state. We are grateful to each of 
these states for providing the data for this research. 


standardized student achievement tests, and student demographics including race, gender, and 
free- or reduced-price lunch (FRL) eligibility status. 

Appendix Table A.1 reports the time period, number of principals and number of students 
for each state. Not surprisingly, the total number of principals differs substantially across states, 
exceeding 11,000 in Texas but below 700 in Massachusetts. The beginning date of the 
longitudinal panel also varies by state, which contributes to this variation. We use the first school 
year in which principals can be linked to schools as the start of the data panel—for most states 
this takes us back to the 1990s. We also restrict the data panels in several ways to make them 
consistent across states. Specifically, the last school year of the panels is 2014-15 in all states 
(the most recent, common year of cleaned data). We restrict our analyses to principals of K-8 
schools, which we define as settings where the highest grade is less than or equal to 9" grade 
(e.g. non-high schools); high schools are excluded because we cannot estimate value added for 
principals in high schools in all states given the lack of year-over-year high school state 
assessments. Charter schools are excluded for this study because some states have no charter 
schools during the time period of our panel (e.g. Washington only passed a charter law in 2012) 
and others have very few.’ Finally, to ensure that we are capturing individuals whose main job is 
serving as a principal, we restrict the data to principals who serve in a single school for at least .5 
FTE in a given year. 

For some of our descriptive analyses about the pathways to the principalship we use the 
full panel of data in each state. But information about principals and their school assignments 
predates the availability of data necessary to estimate value-added, which require that students 
also be linked to schools and that there be year-over-year testing in the same subject. Thus, for 


> Where available, we use data on charter school employment to determine the amount of prior employment 
experience and tenure of principals. 
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the value-added analyses we use a subset of the data in each state that includes just the time 
period over which reliable value-added models can be estimated. 

Table 1 provides comparisons of achievement, the structure of schools and districts, 
student demographics, and changes in population for our six states. Panel A shows means and 
standard deviations for 8 grade NAEP scores in mathematics and reading that illuminate 
substantial differences across both dimensions. First, average NAEP scores are much higher in 
Massachusetts than all other states; they exceed the next highest state by about 25 percent of a 
standard deviation in math and 20 percent in reading. The differences in achievement for the 
other states tend to be far smaller and the rank ordering of the remaining states differs by subject. 
Importantly, these scores reflect myriad family, school and community influences and do not 
indicate differences in school quality. Second, the range of the standard deviation in mathematics 
across states is almost twice as large as the range in reading. The smaller standard deviation in 
Texas NAEP mathematics scores suggests that a one standard deviation move in the standardized 
Texas state test score distribution reflects a smaller difference in actual knowledge than, for 
example, a one standard deviation change in the standardized Washington state test score 
distribution (which has the largest standard deviation). 

Panel B shows the number of school districts, number of schools, school size, and 
enrollment share by district size, where a threshold of ten thousand students divides small and 
large districts. Differences among the number and size of districts illuminate striking differences 
in administrative structures across states that almost certainly affect the structure of the principal 
labor market. On the one hand, over 90 percent of the districts in Texas, Missouri and 
Massachusetts have fewer than 10,000 students. On the other hand, one third of North Carolina 


and almost 20 percent of Georgia districts have enrollment that exceeds 10,000, and the fewer 


number of districts creates relatively larger administrative units. Most of North Carolina’s 115 
districts, for example, are geographically large and county wide. Georgia schools tend to be 
much larger and Washington and Missouri schools much smaller than other states. This likely 
affects the use of assistant principals and potentially the structure of the principal pipeline—e.g., 
district internal labor markets might be relatively more important in the states with fewer and 
larger districts. 

Panel C presents percentiles of student demographics at the school level in 2014-15, and 
here too we observe large differences across states. For instance, consistent with much higher 
NAEP scores, the Massachusetts distribution of share eligible for a subsidized lunch lies to the 
left of the other states. While the 25" percentile school in Massachusetts has only 17 percent of 
low-income students, the shares of students at the 25" percentile is more than twice as high in all 
the other states. The difference at the 50" percentile is smaller but still sizeable.* There are also 
large differences in racial and ethnic diversity. Black enrollment shares are much smaller in the 
non-southern states than in North Carolina and Georgia. Washington in particular has only a 
small number of schools with even a 5 percent black enrollment share. Hispanic enrollment is 
especially low in Missouri, while the median school in Texas is almost 50 percent Hispanic. 

Finally, Panel D of Table 1 shows changes in the number of K-8 schools and K-8 
enrollment between 2004-05 and 2014-15. These changes have important implications for the 
demand for new principals in public schools. Enrollment and growth in the number of schools 
are much higher in Georgia and Texas than in the other states, and enrollment and growth are 


declining over the course of our data panel in Massachusetts. 


3 Note that the upper end of the distribution of students eligible for subsidized lunch likely overstates the level of 
poverty as the Community Eligibility Provision provides that all students in high-poverty schools can receive free ro 
reduced price lunch even if they would not individually qualify. 
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2.b. Principal Eligibility Requirements 

A recent Education Commission of the States report (Scott, 2018) indicates that all states 
have adopted standards to guide school leadership policies. This can entail the requirement of: 
specific types of preparation and training in leadership certification programs;* minimum hours 
of supervised field experiences prior to certification; achieving a minimum GPA; and while not 
universal, most states require that principals have a master’s degree, have at least some prior 
experience as a teacher, and pass one or more certification tests. 

While at a high level states appear to have similar requirements for principal licensure, 
there are differences beneath the surface (see Appendix Table A.2). For example, all six states 
currently require between two and three years of prior experience in schools or the education 
system generally, a valid teacher’s certificate, and, with the exception of Washington, a passing 
score on any standardized exam adopted by the state board. That said, alternative routes to 
administrator/principal certification are available to those who hold a Bachelor’s degree but did 
not complete a traditional certification program in Massachusetts, Missouri, North Carolina, and 
Washington. Washington’s alternative routes also apply to other specific populations, including 
paraeducators with associate’s degrees. A leadership preparation program is also required by all 
five states (Missouri even specifically requires the completion of a course in 
“psychology/education of the exceptional child’ in addition to the preparation program). 

Additionally, there is some degree of flexibility around the master’s degree requirement 
across the six states. For example, Massachusetts has the most flexible standard, allowing for 
either the completion of an approved master’s program with a supervised practicum, completion 
of an apprenticeship, or approval through a panel review. On the other hand, North Carolina is 


4 For instance, degrees in educator preparation programs that are aligned with standard outline by the Council for the 
Accreditation of Educator Preparation. 


more stringent, requiring that the individual’s master’s degree be from a public school 
administration program. Otherwise, the individual must also complete a public school 
administration program meeting the established standards in addition to their master’s degree 
from an accredited college or university. Notably, North Carolina also has more pronounced 
distinctions between Assistant Principals and Principals with respect to the expectations, 
responsibilities, and requirements of the position. Along with seven other states, North Carolina 
invests in the professional development program, AP Ready, which prepares assistant principals 
for the demands of the Principal role and is tailored to fit regional needs (New Leaders, 2018). 

Many of the standards described above were in place prior to the passage of the Every 
Student Succeeds Act (ESSA), which shifted considerable control of the public education system 
to state and local governments but required states to submit plans to the federal goverment 
outlining, among other things, the state’s system of certification and licensing (McGuinn, 2016). 
ESSA, however, gives “unprecedented recognition” for the role of principals and urges the 
implementation of effective principal recruitment as well as preparation and ongoing 
professional learning (NAESP). States implemented some changes (see Appendix Table A.2) 
and accordingly, ESSA also likely influenced the training of principals. For example, 
Massachusetts has made it a priority to expand the pipeline of qualified principals in the school 
system and reduce waivers of requirements.° 

Still, on the whole, there appears to be little legislative reform of requirements for new 


principals in our focus states over the timespan of our analysis:° a search on principal 


5 In Massachusetts, a principal candidate can have his or her requirements waived by the commissioner, which leads 
to educators being placed in positions for which they are not certified. 

® The only notable exception is a Washington House Bill in 2002 that added the requirement of candidates to have 
held a valid teacher or educational staff associate certificate and demonstrated school experience (see Appendix 
Table A.2). 


requirements in all six states’ administrative code in the last 20 years revealed no significant 
legislative changes related to principals.’ This is consistent with a report by the National Center 
for Education Statistics (Hill, et al., 2016) studying principal characteristics in 1987-88 
compared to 2011-2012, which finds little in the way of change in principal demographics over 


this time period.® 


BE Descriptive analysis of principal characteristics and prior experience 

The section describes the distributions of principals by tenure, experience as a principal, 
and prior experience as an assistant principal and teacher, by district size and school 
demographic characteristics across states. All tables report distributions for the 2014-2015 
academic year and we have structured tenure and experience categories to account for the fact 
that many principal spells and educator careers are left-censored. In the most states, we observe 
at least eleven years for a sample window that begins in academic year 2003-2004; our top 
experience and tenure categories in these states therefore capture principals with eleven or more 
years. For states with shorter windows the top tenure/experience categories captures individuals 
with six or more years. We also assume that a principal in her first year in a school in 2014-2015 
did not previously serve as a principal if we do not observe her in another principal position 
during the sample period. Violations of this assumption will lead to the underestimation of 


experience, but this likely only relates to a very small number of cases. 


TIt is possible, however, that there are more significant changes in regulatory interpretation of state laws, 
unfortunately this is more difficult to track. 

8 The report found that across the two periods, only one to three percent of public school principals had a bachelor’s 
degree or less. The percentage of those with a master’s degree was 62 percent in 2011-2012 compared to 53 percent 
in 1987-1988. Likewise, public school principals in 2011-2012 had only about one less year of teaching experience 
than in 1987-88. 


Because the data do not report separate experience levels by position, we cannot be sure 
of educator roles in the years prior to the start of our sample period. We assume that someone in 
a teaching position in the earliest year of the data panel did not serve in either an assistant 
principal or a principal position previously—i.e., we assume the individual’s prior experience is 
in teaching only. Finally, we exclude principals who are in the first year of our data panel in each 
state with an experience level above one (because our data do not allow us to determine the 
nature of the individual’s prior experience). 

3.a. Experience and tenure 

Table 2 shows principal experience and tenure across the six states, where experience is 
defined as years as a principal in any state public school including charter and high schools and 
tenure is defined as the number of years in the current spell as a principal. The distribution of 
experience is strikingly similar across states, and lines up with national figures showing that 
principals tend to have 6-7 years of experience on average and median spells of roughly 4 years.’ 
Note that experience is not higher in Massachusetts despite the declining number of traditional 
public schools in that state. '° 

There is variation in the tenure distribution, as a principal in Georgia and to a lesser 
extent Missouri is less likely to be in her first or second year in a school than a principal in North 
Carolina, Texas or Washington. Interestingly, Georgia has a low share of principals in the first or 
second year despite experiencing substantial growth in the number of schools in the state. 
Principals in Missouri and Washington are especially likely to have long tenure: almost 10 


percent have served at least eleven years in the current position. 


° See Table 6 of the National Teacher and Principal Survey administered by the National Center for Education 
Statistics, https://nces.ed.gov/surveys/ntps/tables/Table_6 042717.asp. 

'0 As discussed above, Massachusetts has the shortest panel of the states in our sample, and we only consider 
whether principals have 6 or more years of principal experience and tenure. 
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Table 3 examines variation across states in experience patterns by proportion eligible for 
a subsidized lunch, proportion black, proportion Hispanic, and average student achievement. 
Note that these are not quartiles of the distribution but rather groups divided by specified shares 
of students; we organize the table this way in order to compare distributions among schools with 
similar demographic compositions in the six states.'' The left panel reveals sharp differences in 
the extent to which principals with little experience are concentrated in high poverty schools. In 
neither Georgia nor Washington does the probability of having a principal in her first or second 
year increase monotonically with share eligible for a subsidized lunch; it lies between 0.20 and 
0.23 for all categories in Georgia and between 0.23 and 0.29 for all categories in Washington. In 
the other 4 states it does rise monotonically by poverty rate, and the differences are particularly 
large in Massachusetts and Missouri where the probability of having a principal in her first or 
second year is roughly one third in schools with at least 75 percent of students eligible for a 
subsidized lunch but less than twenty percent in schools with less than 25 percent eligible. 

Patterns by race and ethnicity diverge from those by income, being much more 
pronounced in Texas and North Carolina. The probability of having a principal in her first or 
second year exceeds one third in North Carolina and approaches 50 percent in Texas in schools 
where black enrollment exceeds 75 percent of the total. Although the probability rises 
monotonically in Georgia, differences by the black enrollment share are much smaller than in 
Texas or North Carolina, particularly in very high proportion black schools. Finally, the states 
also diverge in the strength of the association of principal experience and Hispanic enrollment 
share. On the one hand, the ordering is quite strong in Massachusetts, while on the other hand it 


is quite weak in Texas. The marked differences between these states in the overall level of 


'! Information on the percentile distribution is available from authors upon request. 
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Hispanic enrollment, the association between Hispanic ethnicity and income, and the distribution 
by country of origin may all contribute to these differences. 

Finally, the strongest and most consistent associations between the probability of having 
a principal with little experience and student composition emerges along the average test score 
dimension. With the exception of Washington, the probability of a principal with little or no 
experience declines monotonically with test scores.'? Particularly large differences emerge in 
Massachusetts and Missouri, the two states with the largest differences by share eligible for a 
subsidized lunch. 

3.b. Prior assistant principal, teaching and principal experience 

The next two tables , Tables 4 and 5, report distributions of years of experience as a 
teacher and assistant principal, respectively, while Figure 1 illustrates distributions of principals 
by whether they obtain any assistant principal or teaching experience in a state public school, and 
if so, whether the experience was obtained in the same school, same district but not in the same 
school, or only in another district. 

Panel C of Table 4 for all districts shows pronounced differences in the distribution of 
prior teaching experience across states. The share of principals with 0 or only one year of 
experience exceeds 35 percent in Massachusetts and 20 percent in North Carolina but is below 
10 percent in Missouri, Texas and Washington. In Texas in particular, less than 4 percent of 
principals have fewer than two years of teaching experience, consistent with the state 
requirements to become a principal. Although Panels A and B show that the probability of 
having little or no teaching experience tends to be slightly larger in the larger districts, the 
pattern holds regardless of district size. 


'2 Student test scores are normalized to be mean zero with a standard deviation of 1 within each grade, subject (math 
and ELA/reading), and year. 
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Given the standard view that principals should be qualified to serve as instructional 
leaders, we were surprised to find that in three states (MA, NC and WA) more than 10 percent 
of the principals apparently had zero years of public school teaching experience in the same state 
in which they served in a principalship. Note that our figures may understate teacher experience, 
however, because the experience may have taken place prior to our sample periods and our data 
do not include any teaching experience that principals may have had in private schools or in 
states other than the one in which we are observing their employment as principals. !? In addition, 
some principal spells likely started prior to teaching requirements, '* and in some cases, the state 
might have waived the teaching requirement. 

Turning to Table 5, which reports the proportions of principals with experience as an 
assistant principal, we find, contrary to our expectations, that being an assistant principal is not a 
stepping stone for the majority of principal in four states - Georgia, Massachusetts, Missouri and 
Washington - where at least 40 percent of principals had no assistant principal experience. !° By 
comparison, only 13 percent of principals had no assistant principal experience in North Carolina 
and Texas, where the majority of principals had at least three years of such experience. 

In contrast to the case of teaching experience, consistent differences emerge in the 
probability of little or no assistant principal experience by district size. In all six states this 
probability is higher in small districts. The gap is roughly 10 percentage points in most states, 


though it exceeds twenty percentage points in Texas. 


3 In Massachusetts, for example, we find that about half of principals who are not observed with teaching 
experience but can be matched to a teaching license. This suggests, but does not necessarily imply, that they have 
teaching experience that we cannot observe. 

4 Though examination of each state’s recent legislative history in the last 15-20 years did not reveal any significant 
changes in the principal eligibility requirements in our focus states. One exception is a Washington House bill 
explained in Appendix table A.2. 

> Though, as is the case with teachers, we may miss assistant principal experience occurring prior to the first year in 
the panel or in private schools or another state. 
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Figure 1 summarizes the salient differences in the joint distributions of teaching and 
assistant principal experience by experience location and district size.!° Experiences are lexically 
ordered with assistant principal experience prior to teaching experience. This means that the 
figure ignores teaching experience for those with assistant principal experience, and then reports 
the distributions of teaching experience by location. The third bar for each state shows the 
fraction of principals with no prior assistant principal or teaching experience in the state public 
system. 

We begin with a detailed discussion of the experience distribution for Texas in Panel A to 
clarify the structure of the figure. The left bar illustrates that 86 percent of principals have 
assistant principal experience, with 19 percent having worked as assistant principals in the same 
school (blue rectangle), roughly half in the same district but not in the same school (orange 
rectangle), and around 15 percent in another Texas district (grey rectangle). Out of the 13 percent 
of principals with no prior assistant principal experience, | percent had no teaching experience 
and most of the remainder had teaching experience in the same district but not the same school. 

Comparisons among the states in the top panel produce patterns consistent with both the 
marginal distributions reported in Tables 4 and 5 and the administrative structures discussed in 
Section 2. Internal labor markets seem particularly important in Georgia and North Carolina, two 
states with relatively large districts. In these states, the vast majority of principals accumulated 
experience in the same district. By comparison, principals in Washington and Massachusetts, 
two states with relatively large numbers of smaller districts, were more likely to gain experience 


in other districts. 


'6 Appendix Table A3 presents the full joint distributions. 
14 


The probability that a principal obtains assistant principal experience in another district is 
higher in smaller than in larger districts. This makes sense given the larger number and 
geographic proximity of district schools. It is also not surprising that the probability of having 
obtained experience in the same school is higher in smaller districts where the school constitutes 
a larger fraction of the internal labor market. 

Table 6 presents the final component of the descriptive analysis of the distribution of 
principal characteristics. It shows the fractions of principals who previously served in a principal 
position at a different school in the same district and in a different district. The bottom panel of 
the table reveals pronounced differences in the share with previous principal experience, ranging 
from over one third in Washington, to roughly 30 percent in Missouri, North Carolina and Texas, 
to less than 20 percent in Georgia and Massachusetts. There are also differences in where 
principals previously worked. In North Carolina, Georgia, and to a lesser extent Texas, principals 
were much more likely to lead another school in the same district, while in Massachusetts 
principals were much more likely to have led a school in a different district. Principals in 
Missouri and Washington are roughly equally divided in terms of whether they previously led a 
school in the same or in a different district. A comparison of the top two panels shows the 
expected pattern that principals in small districts were more likely than those in a large district to 


obtain any previous experience as a principal in another district. 


4. Within-school differences in principal value added 
The importance and policy relevance of differences in principal characteristics and prior 
experiences hinges largely on the importance of leadership quality for student outcomes. This 


section examines patterns of achievement differences across principals who work in the same 
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school during the sample period. It begins with the development of a conceptual framework in 
which to consider differences in achievement growth, highlighting factors that complicate the 
interpretation of differences in value-added across principals within schools. We compare the 
identification of principal effects with those of teachers and discuss impediments to the 
measurement of principal effectiveness, our approach to mitigate these impediments, and our 
interpretation of the resulting estimates. Following this discussion, we present two sets of value- 
added estimates. The first aims to recover estimates of the within school variation in principal 
value added across states, and the second aims to assess differences in value added by 
experience, tenure and highest degree earned for each state. We focus on mathematics 
achievement in this section because research shows that school-based factors tend to have 
stronger effects on mathematics than reading. '’ Note that because states administer different test 
instruments the estimates do not support direct comparisons across states. 
4.a.__ Analytic Approach 

We first describe the specifications used to estimate the within-school variation in value 
added across principals and the relationship between value added and principal tenure, 
experience and other characteristics. We outline key issues that complicate the estimation and 
interpretation of the differences in value added prior to describing the specifics of the regression 
models. 
4.a.1. Within-school variance 

A growing body of literature estimates teacher effects on learning and achievement, and 


it is possible to apply similar methods to estimate effects of principals.'’ However, there are 


'7 Following on the literature on teacher value-added, we expect variation in principal value-added to be larger in 
mathematics than in reading (Hanushek and Rivkin, 2010; Koedel, Mihaly, and Rockoff, 2015). 

'8 As highlighted in Branch et al. (2018), estimation of principal value added must address many of the same but 
also some very different issues as estimation of teacher value added. A growing body of literature generates 


16 


some different challenges associated with the identification of principal effects that raise 
questions about interpreting these estimates as causal measures of principal quality. The most 
fundamental of these is that it is difficult to separate the influence of the principal from other 
factors associated with the school. Estimation of principal effects is essentially equivalent to the 
estimation of average school effects during all or a specified portion of a principal’s spell at a 
school. Consequently, myriad factors out of the control of the principal, such as school and 
neighborhood shocks, pre-existing relationships amongst teachers, will influence the estimates. 

Another issue is that the influence of one principal may transcend his or her spell at a 
school. For instance, principals can influence school quality even after leaving through teacher 
hiring or the establishment of school norms and culture. Controls for prior achievement account 
for effects of the school prior to the entry of the new principal, but they do not account for 
influences of the prior principal that persist following her departure. Consequently, although 
differences in value added between years in which a school is led by different principals will 
reflect differences in principal productivity, they also capture other factors and should thus be 
interpreted with care. The concept of principal value added as used in this analysis refers to value 
added during all or part of a principal’s spell at a school. 

Our procedure for estimating principal value added closely follows the teacher value 
added literature. We aim to isolate within-school differences in principals over time. First, we 
estimate the following value added model of student achievement: 


(1) Aisnpt = f (Aist-1) + Xise + Sse + On + d, a3 Ons + Ejisnpt 


estimates of principal effects and considers the methodological impediments. This includes: (Clark and Martorell 
2009; Branch, Hanushek, and Rivkin 2012; Chiang, Lipscomb, and Gill 2012; Coelli and Green 2012; Hochbein and 
Cunningham 2013; Dhuey and Smith 2014; and Grissom, Kalogrides, and Loeb 2015). 
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Equation (1) models outcome (A) for student 7, in school s, in grade A, with principal p, and in 
year ¢t as a function of individual, school, and principal factors. f(Aj,¢-1) is a cubic polynomial 
of prior year standardized test scores in math and reading. X;,; is a vector of student controls 
that include indicators for gender, ethnicity, free and reduced-price lunch eligibility, special 
education, and whether the student is in her first year at school s due to a non-structural move 
(i.e. it is not the first grade offered in the school). The vector S,, consists of school averages of 
the student variables in X;,,. The terms 6; and ¢, are indicators for grade and year, respectively. 
Oy, 1s a principal-by-school fixed effect, and €j;,; 18 an error term clustered at the school level. 
To minimize the influence of any turbulence around principal transitions, we exclude the first 
and last years of all principal spells. 

6, contains information about principal value added but is likely confounded by other 


school factors per the discussion above. To remove the influences of fixed school factors, we 


demean 6,, within schools by subtracting the school-average value, 0,,, where 0,, = 


yee Tp Ops, Mp 1s the ratio of years principal p leads school s to the total number of years school 


s appears in the data panel, and P, is the number of principals who served at school s over the 
course of the data panel. We denote the demeaned values as 0, = Oy; — 6, s» Where 6,,, can be 
interpreted as the difference in school performance during the tenure of principal p relative to 
school performance during the tenure of other principals at the same school 

Note that our focus on principal by school fixed effects differs from studies such as 
Grissom et al., (2015) and Chiang et al (2016) that estimate principal fixed effects using models 
that also include school fixed effects. Those specifications include a single indicator for each 


principal, and appearance of a principal in multiple schools creates linkages among schools that 


foster comparisons of all principals who share the same connected network as described in the 
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aforementioned papers.'? By comparison, we estimate a separate effect for each principal spell in 
a school and focus on comparisons across principals in the same school. This is consistent with 
the notion that these estimates capture differences in school average value added between 
periods in which a school is led by a different principal. Although the estimates capture principal 
influences, the complex dynamics and contributions of other factors precludes the interpretation 
of within-school differences as measures of differences in principal effectiveness. 

The demeaned principal-by-school effects from equation (1) can be used to recover the 
distribution of within-school principal value added. The variance of this distribution is 
informative about the potential importance of principals in determining student achievement. For 
example, if there is no variance, it would suggest that differences in principals do not contribute 
to student achievement growth, or put differently, that the assignment of specific principals is not 
consequential for student test scores. 

The raw variance of 6;, will overstate the true variance of principal value added because 
of sampling variance. To address this problem, we will develop a randomized-inference 
procedure by which the sampling variance of @,,, can be estimated, and subsequently subtracted 
from the raw variance estimate, leaving only variance that can be plausibly attributable to 
principals. This will be operationalized in the subsequent draft; the variances reported in this 


paper include the error variance. 


'? As documented by Chiang et al. (2016), a complication with this approach is that unlike for teachers, networks of 
principals across schools are thin and there are many broken connections. Correspondingly, the variance of principal 
value-added estimated by this approach is not directly interpretable as either the total variance or the within-school 
variance. Rather, it is the within-network variance, which should be more than the within-school variance alone if 
there is cross-network variance, but less than the total variance. 
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4.a.2. Differences by principal characteristics 

In addition to assessing the overall importance of principals as measured by the within- 
school variance of principal fixed effects, we are also interested in whether specific principal 
characteristics are associated with achievement growth. To examine this question we use a model 
akin to what we show in equation (1), but we replace the principal-by-school fixed effects with a 
vector of principal qualifications and attributes (P,;). We also add school fixed effects (y,) to the 
model, preserving our fundamental within-school identification strategy for recovering the 
parameters of interest: 

(2) Aisnpt = f Aist-1) + Xist + Sse + Por + On + 6, + Vs + isnpe 

The vector P,; contains indicators for highest degree earned by principal p in year t, along with 
categories of principal tenure and experience. The categories of principal tenure and experience 
included in the regression are 3, 4, 5, 6 to 10, and 11 or more years, relative to the holdout group 
of 2 years (recall that first-year principal spells are omitted from the data). €jspp¢ is an error term 
clustered at the school level. 
4.b. Results 

Table 7 reports estimates of the standard deviation of the distribution of principal value- 
added for each state overall, and separately for small and large districts. As noted above, we 
focus on principal value-added to math achievement. Also recall that our estimates are 
constructed to isolate within-school variation in principal value-added only, as shown in equation 
(1). Any cross-school variation is necessarily omitted. 

As noted above, the standard deviations in Table 7 are unadjusted for sampling variance 
at present, which surely leads to an overstatement of the true variability in principal value-added. 


Noting this caveat, the estimates suggest the potential for meaningful differences in principal 
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value-added within schools. The estimates also suggest there are differences across states in the 
amount of variation in principal value-added. Specifically, the results indicate that differences in 
principal value-added are most important in Texas and North Carolina, and least important in 
Missouri. Some of these differences may be attributable to contextual factors across states—for 
example, the much smaller variance of NAEP scores in Texas suggests that a within-school 
standard deviation based on the state standardized test corresponds to less variation in underlying 
knowledge than in the other states. Consequently, the larger estimates for Texas may not actually 
correspond to larger effects on the actual variation in knowledge gained. That said, to the extent 
that the differences across states persist after adjustments for sampling variance and cross-state 
differences in context, they would motivate further inquiry into why there is more variability in 
principal value-added in some states relative to others. A potential explanation is cross-state 
variation in the regulations surrounding principal certification—for example, all else equal, states 
that permit more diverse pathways into principalship should expect a more heterogeneous pool 
or principals. 

Next, in Table 8 we report on the extent to which principal experience and education are 
predictive of within school estimates of principal value-added, per equation (2). We focus on the 
two measures of principal experience described above: tenure at the current school and overall 
experience. Although these two measures are correlated, they are sufficiently distinct to permit 
separate identification. Because we omit the first year of each principal spell at a new school 
from the analytic sample, the first observed year of principal experience or tenure is year-2. We 
use year-2 as the omitted comparison group, which means the other coefficients are estimated 
relative to principals in their second year (either at a school or in terms of overall experience). 


Because previous research suggests that the returns to principal experience plateau quickly 
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(Clark, Martorell, and Rockoff, 2009), we differentiate experience year-by-year early in the 
career, but after year-5 we group principals more coarsely by whether they have 6-10 years of 
experience/tenure, or 11 or more years. 

There is variation in the education credentials that principals carry across states, 
specifically with respect to whether “education specialists” are observed in principal positions. 
For the sake of consistency in Table 8, in each state we omit principals with master’s degrees and 
compare all other education categories to that group. (Note that the Massachusetts data only 
includes information about educational attainment for in-state program completers. As such, we 
have not included educational attainment in the model estimated for Massachusetts.) 

Focusing first on the results for tenure in the school and overall experience, there is not 
consistent evidence that either of these measures is systematically related to principal value- 
added. Most of the estimates for the tenure and experience gradients are small and statistically 
insignificant, with exceptions for tenure in North Carolina and overall experience in Texas and 
Washington. We do not draw strong inference from these selected results because they do not 
replicate across states and are inconsistent in sign; moreover, given the large number of tests we 
perform in the various states, some spurious significant correlations would be expected. Overall, 
we conclude from Table 8 that principal tenure in the school and overall experience are not 
consistently related to student achievement growth.”° 

Finally, we turn to the estimates that compare the education categories. There is also no 


consistent evidence that achievement growth is related to the education level of the principal. 


20 Clark et al. (2009) find that schools have higher achievement when they are led by more experienced principals in 
New York City, which is similar to what we find in Texas but not in the other states. Clark et al. (2009) find the 
largest jump in achievement occurs between experience years | and 2, which we do not compare in our study 
because we drop all year-1 principal spells at the onset, as described above—this may help to explain the 
discrepancy. More broadly, our ability to test the relationship between experience and student test score growth 
across multiple states simultaneously using common methods raises our confidence in the results. 
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The one significant coefficient is in Washington for principals with a bachelor’s degree (again, 
relative to a master’s degree), but this estimate is based on a very small sample of principals and 
it is not buttressed by the estimates in the other states. Similarly to tenure and experience, we 
conclude from Table 8 that differences in principals’ education levels are not important 


predictors of value-added. 


5. Summary and Policy Considerations 

This paper describes the distributions of principals along a number of dimensions across 
six states to get a better understanding of differences in the markets for school leaders and the 
potential contributions of principals to observed variation in student achievement. Although 
distributions of principal experience and tenure appear to be quite similar across the states, larger 
differences emerge in pathways to the principal position. In particular, there is substantial 
variation in the share of principals who previously served as assistant principals and in the 
probability that a principal previously worked in the same school or district. Although some of 
these differences come from variation in state population density and urbanicity, differences in 
district structures also seem to be important. Some states have far fewer but larger districts than 
others, and internal markets play a larger role in these states. 

The preliminary achievement analysis is consistent with significant variation in principal 
effectiveness, though more work needs to be done to account for sampling error. Moreover, the 
complicated dynamics of estimating principal effects suggests caution in interpreting the 
estimates as direct measures of principal quality; our measures of variation in achievement 
growth differences among principals who lead the same school therefore provide only suggestive 


evidence of productivity differences. There does not seem to be systematic variation by 
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experience or highest degree earned. This is consistent with research on teachers showing 
substantial variation among educators with similar credentials. 

In terms of policy, state differences in test instruments preclude strong inference from 
cross-state differences in the distributions of value added. State differences in district structures 
seem to play an important role in the determination of the use of external and internal labor 
markets. Greater district reliance on their own educators to serve as principals in some states 
elevates the importance of producing school leaders, while in other states external hiring plays a 
more important role. Perhaps the single largest difference among the states is the probability a 
principal has assistant principal experience, and the value of such experience merits additional 


consideration. 
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Tables and Figures 


Figure 1. Proportion of principals with prior experiences as an assistant principal, and for 
those with no prior experience as an assistant principal, as a teacher, by location of prior 
experience and state 

A. All districts 


Distribution of Teaching and Assistant Principal Expereince for AY 2014-15 in All Districts 
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B. Small Districts 


Distribution of Teacher and Assistant Principal Experience in AY 2014-15 in 
Small Districts 
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C. Large Districts 


Distribution of Teaching and Assistant Principal Expereince for AY 2014-15 in Large 


Districts 
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same school same district, not same school only other district none 


Notes: In some states the total of the shares is less than one due to the masking of information derived from small samples sizes. 
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Table 1. Distributions of Student and School District Characteristics in the Six States 
Panel B. Numbers of districts, schools serving K-8 and enrollment share, by 
district enrollment in 2014-15 


Panel A. 2014-15 National 


Assessment for Education Progress 


Math Reading # Districts # Schools Mean School Size __ Enrollment Share 

Mean SD Mean SD Small Large Small Large Small Large Small Large 

dist. dist. dist. dist. dist. dist. dist. dist. 

GA 279 36 262 35 146 34 608 1174 = 5563.2 707.5 0.709 0.291 
MA 297 3939 274 35 395 10 1109 316 417.0 426.9 0.792 0.208 
MO 281 36 267 34 253 20 557 309 398.3 497.2 0.591 0.409 
NC 281 38 261 38 77 38 548 1260 423.4 606.8 0.226 0.774 
TX 284 «633 261 35 1111 108 2060 3565 442.9 704.7 0.266 0.734 
WA 287 = 40 267 36 264 31 816 795 366.0 517.7 0.435 0.565 


Table 1. (Continued) 
Panel C. Percentile cutoffs for student demographic characteristics, for K- 
8 school in 2014-15 


Panel D. 
Change in the number of K-8 
Schools and Enrollment from 
2004-2005 to 2014-2015 


% Economically % Black % Hispanic Schools Enrollment 
Disadvantaged 

25th 50th 75th 25th 50th 75th 25th 50th 75th 

%ile “ile “ile %ile %ile %ile Nile %ile Nile 
GA 0.452 0.649 0.830 0.137 0.326 0.660 0.047 0.088 0.171 6.40% 22.30% 
MA 0.173 0.349 0.668 0.016 0.037 0.098 0.037 0.074 0.239 -4.30% -5.80% 
MO 0.382 0.566 0.741 0.014 0.057 0.171 0.018 0.034 0.065 2.70% 0.50% 
NC 0.435 0.611 0.956* 0.079 0.218 0.410 0.070 0.122 0.208 5.18% 4.96% 
TX 0.436 0.663 0.841 0.015 0.056 0.150 0.240 0.472 0.788 8.04% 13.50% 
WA 0.354 0.554 0.728 0.005 0.015 0.043 0.090 0.147 0.261 1.70% 5.20% 


Notes: Figures are reported from U.S. Department of Education, Institute of Education Sciences, National Center for 
Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics and Reading 
Assessments. Panel A figures represent mean and standard deviation of 8" grade composite scores for all students 
tested in each sample state. *Small and large districts are defined as having student enrollment less than or greater 
than 10,000 for grades K-12. All other statistics are restricted to K-8 settings, as defined by schools that have a 
maximum grade of 9 or less (e.g. excluding K-12 schools). Enrollment share is the proportion of student enrollment 
in the state served by small or large districts. Panel C reported figures are the 25, 50, and 75" percentile of the 
indicated student demographic for each state. These are calculated among “K-8” schools, defined in the previous 
panel, for the 2014-15 school year. Panel D Figures for schools represent the change in the number of K-8 schools 
from 2004-05 and 2014-15, and figures for enrollment represent the similar change in student enrollment for K-8 
schools. 
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Table 2. Distribution of K-8 School Principal Tenure and Experience in 2014-15, by State 


1 2 3 4 5 Serio. 
more 

Panel A. Years of experience as a principal 
Georgia* 0.12 0.11 0.14 0.09 0.09 0.45 with 6 or more * 
Massachusetts? 0.14 0.12 0.10 0.09 0.54 with 5 or more? 
Missouri 0.09 0.14 0.11 0.08 0.08 O32 0.18 
North Carolina 0.14 0.12 0.10 0.08 0.09 0.29 0.18 
Texas 0.15 0.14 0.12 0.09 0.08 0.29 0.14 
Washington 0.14 0.11 0.10 0.07 0.07 0.26 0.25 
Panel B. Years of tenure at current school as a principal 
Georgia* 0.18 0.14 0.16 0.11 0.09 0.32 with 6 or more * 
Massachusetts? —_().22 0.16 0.13 0.11 0.39 with 5 or more? 
Missouri 0.16 0.19 0.13 0.10 0.08 0.25 0.09 
North Carolina 0.24 0.19 0.16 0.10 0.09 0.17 0.05 
Texas 0.22. 0.19 0.15 0.10 0.07 0.20 0.05 
Washington 0.23 0.17 0.13 0.10 0.08 0.20 0.08 


Notes: Individuals in the sample include only principals who work in K-8 schools (defined in Table 1), in the 2014-15 school 
year. Prior experience includes any experience as a principal in the data regardless of sector (e.g. K-8 or high school), and we 
include prior experience as a principal in a charter school where data is available. By construction, all individuals in the 
sample have at least 1 year of experience & 1 year of tenure at current school. We ignore gaps in service, and calculate the 
sum over all years for tenure at current school. We define “pre-service windows” in each state to deal with left censuring of 
experience, and use these to identify whether individuals have 11 or more years of experience or tenure. * Georgia has a 
shorter panel at 9 years, and as such we are not able to classify higher ranges of experience. For this reason we report 6 or 
more years of experience or tenure in the 6" column. ’ Massachusetts has the shortest panel at 8 years, and as such, we are 
not able to classify higher ranges of experience, and so report only individuals with 5 or more years of experience or tenure. 
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Table 3. Share of schools in 2014-15 who have a principal in their first or second year in the principal role, by school 


characteristics and state 


Panel A. Share of schools by student demographics 


% economically disadvantaged % Black % Hispanic 

<=25% 25t0 75% >715% |<=25% 25to 75% >715% |<=25% 25t0o75% >75 % 
Georgia 0.221 0.206 0.229 0.203 0.210 0.255 0.217 0.210 0.190 
Massachusetts 0.195 0.276 0.326 0.253 0.318 0.182 0.229 0.323 0.415 
Missouri 0.137 0.215 0.317 0.219 0.237 0.305 0.224 0.379 a 
North Carolina 0.211 0.251 0.287 0.253 0.265 0.354 0.260 0.268 0.50° 
Texas 0.237 0.286 0.293 0.278 0.301 0.444 0.276 0.283 0.290 
Washington 0.259 0.234 0.286 0.249 0.257 8 0.243 0.270 0.242 


Panel B. Share of schools by average math and reading test scores 


<= 25th 25th to 75th => 75th 

percentile percentile percentile 
Georgia 0.243 0.222 0.178 
Massachusetts 0.359 0.233 0.174 
Missouri 0.297 0.232 0.149 
North Carolina 0.295 0.265 0.220 
Texas 0.334 0.284 0.225 
Washington 0.307 0.223 0.230 


Notes: The columns in Panel A are defined as K-8 schools (defined in Table 1) as either having less than or equal to 25%, 25% to 75%, or greater than 75% of 

their students as the indicated category (e.g. economically disadvantaged, Black) in the 2014-15 school year. The columns in Panel B are defined by percentiles 
of student achievement, normalized to mean 0 standard deviation | by state, year, grade, and subject, and averaged across mathematics and reading. Each entry 
represents the proportion of schools in the given category that have a principal in their first or second year in the principal role. * WA has no schools with more 

than a 75% Black student population. ° MO has a very small number of schools with more than a 25% Hispanic student population. “NC has only 2 school in 


>75% category. 
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Table 4. Distribution of prior experience as a teacher for AY 2014-15 K-8 principals, by 
district size and state 


0 1 2 3005 6to1o 1% 
more 

Panel A. Small Districts (less than 10K) 
Georgia 0.080 * . 0.06 0.246 0.606 
Massachusetts 0.207 0.148 0.141 0.504 with 3 or more * 
Missouri 0.055 * 0.025 0.169 0.463 0.273 
North Carolina 0.141 0.048 0.048 0.229 0405 0.130 
Texas 0.012 0.012 0.022 0.236 0400 0.318 
Washington 0.091 * 0.015 0.136 0.370 0.377 
Panel B. Large Districts (10K +) 
Georgia 0.197 * ig 0.070 0.311 0.403 
Massachusetts 0.193 0.160 0.101 0.546 with 3 or more * 
Missouri G.105:. -* 0.041 0.229 0.373 0.236 
North Carolina 0.190 0.065 0.074 0.276 0.355 0.041 
Texas 0.022 0.010 0.022 0.290 0.424 0.232 
Washington 0.109 0.018 0.030 0.193 0.407 0.244 
Panel C. All Districts 
Georgia O155-. - 0.067 0.288 0.475 
Massachusetts 0.204 0.151 0.132 0.513 with 3 or more * 
Missouri 0.073 0.016 0.031 0.191 0.431 0.260 
North Carolina 0.175 0.059 0.066 0.262 0.370 0.068 
Texas 0.019 0.011 0.022 0.273 0417 0.259 
Washington 0.101 0.014 0.023 0.166 0.390 0.306 


Notes: Individuals in the table include all principals who work in K-8 schools (defined in Table 1) in the 2014-15 
school year who (a) appear after the first year of each state’s panel or (b) appear in the first year of the panel as a 
teacher. For educators appearing in the first year as a teacher, all prior experience is assumed to be teacher 
experience. Prior experience includes any experience as a teacher in the data regardless of sector (e.g. K-8 or high 
school), and we include prior experience as a teacher in a charter school where data is available. Note, unlike Table 
2 on principal experience and tenure, it is possible for individuals to have zero years of experience as a teacher. * 
Massachusetts has the shortest panel at 8 years, and as such, we are not able to classify higher ranges of experience, 
and so report only individuals with 3 or more years of teaching experience. In Georgia, Missouri, and Washington, 
cells that represent less than 10 individuals are masked with an asterisk (*). 
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Table 5. Distribution of prior experience as an assistant principal for AY 2014-15 K-8 


principals, by district size and state 


0 1 2 3to5 6to 10 11 or more 
Panel A. Small Districts (less than 10K) 
Georgia 0.414 0.264 0.155 0.166 * with 6 or more * 
Massachusetts 0.607 0.096 0.134 0.151 * with 6 ormore? 
Missouri 0.590 0.092 0.097 0.169 0.044 : 
North Carolina 0.172 0.141 0.178 0.383 0.115 0.011 
Texas 0.258 0.100 0.133 0.340 0.153 0.015 
Washington 0.540 0.083 0.065 0.198 0.106 : 
Panel B. Large Districts (10K +) 
Georgia 0.435 0.178 0.146 0.218 * with 6 or more? 
Massachusetts 0.664 * 0.118 0.134 * with 6 or more? 
Missouri 0.350 0.147 0.147 0.226 0.124 = 
North Carolina 0.112 0.089 0.155 0.439 0.191 0.014 
Texas 0.079 0.056 0.133 0.468 0.235 0.029 
Washington 0.406 0.133 0.113 0.226 0.113 . 
Panel C. All Districts 
Georgia 0.427 0.210 0.149 0.199  * with 6 or more * 
Massachusetts 0.619 0.092 0.131 0.147  * with 6 or more? 
Missouri 0.505 0.111 0.115 0.189 0.073 ‘s 
North Carolina 0.131 0.105 0.162 0.422 0.168 0.013 
Texas 0.136 0.070 0.133 0.428 0.209 0.025 
Washington 0.468 0.110 0.091 0.213 0.110 0.008 


Notes: Individuals in the table include all principals who work in K-8 schools (defined in Table 1) in the 2014-15 
school year who (a) appear after the first year of each state’s panel or (b) appear in the first year of the panel as a 
teacher. For educators appearing in the first year as a teacher, all prior experience is assumed to be teacher 
experience. Prior experience includes any experience as an assistant principal in the data regardless of sector (e.g. K- 
8 or high school), and we include prior experience as an assistant principal in a charter school where data is 
available. Note, unlike Table 2 on principal experience and tenure, it is possible for individuals to have zero years of 
experience as an assistant principal. * Georgia has a relatively shorter panel at 9 years, and as such we are not able to 
classify higher ranges of experience, and so report only individuals with 6 or more years of assistant principal 
experience. Massachusetts has the shortest panel at 8 years, and as such, we are not able to classify higher ranges of 
experience, and so report only individuals with 6 or more years of assistant principal experience. 
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Table 6. Distribution of AY 2014-2015 K-8 principals by location of prior principal 
experience in another position, by district size and state 


Same district and not Only other 


the same school district None 
Small Districts 
Georgia 0.086 a 0.874 
Massachusetts 0.024 0.153 0.824 
Missouri 0.106 0.197 0.697 
North Carolina 0.242 0.086 0.672 
Texas 0.146 0.143 0.711 
Washington 0.159 0.208 0.633 
Large Districts 
Georgia 0.083 * 0.898 
Massachusetts 0.101 0.084 0.815 
Missouri 0.213 0.162 0.624 
North Carolina 0.329 0.071 0.600 
Texas 0.234 0.081 0.685 
Washington 0.274 0.162 0.565 
All Districts 
Georgia 0.084 0.027 0.889 
Massachusetts 0.040 0.138 0.823 
Missouri 0.144 0.185 0.671 
North Carolina 0.302 0.076 0.622 
Texas 0.206 0.100 0.693 
Washington 0.220 0.183 0.596 


Notes: Individuals in the table include all principals who work in K-8 schools (defined in Table 1) 
in the 2014-15 school year who (a) appear after the first year of each state’s panel or (b) appear in 
the first year of the panel as a teacher. For educators appearing in the first year as a teacher, all 
prior experience is assumed to be teacher experience. Columns indicate whether the prior 
experience as a principal is in the same district that they now serve, prior experience only in a 
different school district, or no prior experience as a principal. The last category also includes 
individuals who may have out-of-state principal experience that is not observed. Each entry 
represents the proportion of K-8 principals in 2014-15 with prior experience indicated by the 
column. In Georgia, cells that represent less than 10 individuals are masked with an asterisk (*). 
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Table 7. Estimated Within-School Variation in Principal Value-Added to Math Test Scores 
in Standard Deviation Units, by District Size and State. 


Massachu- North 
Georgia setts Missouri Carolina Texas Washington 

Small 
districts 0.041 0.085 0.054 0.121 0.179 0.082 
Large 
districts 0.038 0.132 0.079 0.111 0.128 0.071 
All 

fons 0.039 0.097 0.064 0.116 0.149 0.087 
districts 


Notes: The value-added models regress student test scores on a cubic polynomial of prior year achievement, student 
indicators for gender and race/ethnicity, an indicator for whether the student is in their first year in the school, and 
indicators for participation in LEP, SPED, and FRL programs. We also include school-averaged versions of these 
variables, year and grade indicators, and principal-by-school fixed effects. After estimating the models we demean 
the principal-by-school fixed effects by school means. School means are the weighted average of principal-by- 
school fixed effects, weighted by the years of service for each principal. 
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Table 8. Within-School Differences in Principal Mathematics Value Added by Tenure, 
Experience, and Education, by State 


North 


Georgia § Massachusetts Missouri . Texas Washington 
Carolina 
Tenure at Current School 
eee 0.029 0.012 0.018** 0.008 -0.006 -0.004 
: (0.035) (0,008) (0.009) (0.008) (0.006) (0.006) 
4 -0.024 0.015 0.007 -0.020*** -0.009 0.004 
ears 
z (0.045) (0.014) (0.011) (0.007) (0.007) (0.008) 
5 -0.064 0.010 -0.007 0.001 -0.013 0.001 
ears 
a (0.066) (0.019) (0.012) (0.007) (0.008) (0.009) 
6-10 -0.094 0.024 -0.003 -0.011* -0.010 0.001 
- ears 
(0.080) (0.026) (0.013) (0.006) (0.008) (0.011) 
soil tea e . -0.010 0.038** -0.013 -0.011 
(0.020) (0.015) (0.019) (0.023) 
Principal experience 
eee 0.006 0.006 -0.007 -0.025 0.008 -0.016 
‘ (0.035) (0.037) (0.010) (0.015) (0.006) (0.035) 
rome 0.040 -0.045 -0.005 -0.012 0.011 -0.059+ 
i (0.43) 0,034) (0.012) (0.015) (0.008) (0.034) 
ee 0.055 -0.024 0.020 0.002 0.017** -0.089* 
2 (0.059) (0,034) (0.013) (0.014) (0.008) (0,041) 
ee 0.093° 2,019 0008 0.009 0.014# 008 
(0.075) (0.036) (0.014) (0.013) (0.008) (0.021) 
0.018 -0.014 0.006 -0.020 
= * * c 
ten (0.019) (0.013) (0.015) (0.019) 
Highest degree earned 
pacneioe : 0.016 0.006* -0.011 -0.221** 
achnelors 
(0.023) (0.004) (0.009) (0.081) 
Seay 0.015 -0.003 0.007 r * 
p (0.047) (0.010) (0.007) 
Aieiae 0.057 0.007 0.003 0.011 0.043 
(0.048) (0.011) (0.014) (0.013) (0.027) 
Observations 314,980 538,438 1,052,578 1,465,443 3,606,578 947,199 


Notes: The value-added models regress student test scores on a cubic polynomial of prior year achievement, student 
indicators for gender and race/ethnicity, an indicator for whether the student is in their first year in the school, and 
indicators for participation in LEP, SPED, and FRL programs. We also include school-averaged versions of these 
variables, year and grade indicators, and school fixed effects, in addition to the focal indicators for principal tenure, 
experience, and degree type. Note that not all states have education specialists working as school principals—states 
without specialists have that cell blank. Also note two limitations of the MA data panel for this analysis; (1) the MA 
data do not include information on the education levels of principals so these variables are omitted from the model 
using MA data, and (2) the MA data panel is not long enough to identify principals in the highest tenure/experience 
categories (11+ years). Similarly, the Georgia panel is not long enough to identify principals in the highest 
tenure/experience categories (11+ years). In Georgia, cells that represent less than 10 individuals are masked with an 
asterisk (*). 
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Table A.1. Sample Years and Observations Across All States 


Employment Data Value Added Sample Data 
Principal-Y ear Student- Year 
Observations Observations 
From To From To (Unique Principals) | (Unique Students) 
} 1010 314,980 
Georgia 2006-2007 2014-2015 2008-2009 2014-2015 (401) (203,555) 
2055 538,438 
Massachusetts 2007-2008 2014-2015 2009-2010 2014-2015 (664) (298,647) 
; ; 4595 1,052,578 
Missouri 1991-1992 2014-2015 2006-2007 2014-2015 (1252) (528,119) 
: 18,677 3,809,076 
North Carolina 1998-1999 2014-2015 1998-1999 2014-2015 (4,449) (1,908,307) 
49,135 9359525 
Texas 1994-1995 2014-2015 1995-1996 2014-2015 (11,431) (4,936,073) 
: 3791 947,199 
Washington 1983-1984 2014-2015 2006-2007 2014-2015 (906) (485,986) 


Notes: Each panel uses the earliest available year of data, and all panels are normalized to end in 2014-15. Principal observations are unique counts of individuals 
identified as serving as a principal in any year. Student observations are restricted to available ranges of achievement data, and do not necessarily comport with 
the length of the employment panel. 
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Table A.2. Requirements for principal role and major reforms across sample states 


minimum 


2018 Requirements Major Reforms in Last 15-20 Years 
Practicum Prior Waiver of Every Student Succeeds Act Consolidated State Plan (effective 2017) and Other 
Requirement Experience Requirements? Legislation 
GA 750 hours Yes, no Not specified | Four-tiered certification structure adopted in 2014. Principal candidates must earn an 


Educational Leadership — Tier II certificate. a 


specified 


MA Minimum 3 years Yes Implemented the Massachusetts Tests for Educator Licensure, which is designed to align 
500 hours with the subject matter knowledge requirements for educators. » 
MO Minimum 2 years Not specified | Paths toward certification: traditional (bachelor’s degree in some education field earning 
300 hours an initial certificate), alternative (bachelor’s degree in a different discipline, return to a 
college of education and teach simultaneously to earn initial certificate), temporary 
authorization (bachelor’s degree in another discipline, take self-directed courses and teach 
under a mentor; pass exit examinations and work under a one-year renewable certificate to 
ear initial certificate), ABCTE (bachelor’s degree, meet ABCTE requirements and be 
issued Initial Professional Certificate). . 
NC Yes, no 3 years Not specified | No additional changes were made to the licensing protocol. 4 
minimum 
specified 
TX Yes, no 2 years Not specified | No additional changes were made to the licensing protocol. ¢ 
minimum 
specified 
WA Minimum 3 years Not specified | Will develop, improve, and implement programs that establish, expand, or improve 
540 hours alternative routes for certification, as well as mechanisms for recruiting and retaining 


school leaders. ¢ 


Substitute House Bill 2415 (2002): In addition to the administrative certificate, the 
amendment requires candidates to have held a valid teacher or educational staff associate 
certificate and demonstrated school experience. , 


Notes: Information retrieved from https://www.ecs.org/50-state-comparison-school-leader-certification-and-preparation-programs/ 


a https://www.gapsc.com/Rules/Current/Certification/505-2-.153.pdf?dt=%3 C%#Eval(‘strTimeStamp')%20%%3E 


» http://www.doe.mass.edu/federalgrants/essa/stateplan/ - https://dese.mo.gov/sites/default/files/ESSA-Plan-Final.pdf 


ahttps://www2.ed.gov/admins/lead/account/stateplan17/ncconsolidatedstateplan.pdf 
ehttps://tea.texas.gov/About_ TEA/Laws and Rules/ESSA/Every Student Succeeds Act/ 


rhttp://www.k12.wa.us/ESEA/ESSA/pubdocs/ESSAConsolidatedPlan-Final.pdf?_sm_au_=iVVw1VFTFRRvqwQH 


ghttp://lawfilesext.leg.wa.gov/biennium/2001-02/Pdf/Bills/Session%20Laws/House/2415-S.SL.pdf?cite=2002%20c%2078%20%C2%A7%201; 
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Table A3. Distribution teaching and assistant principal experience for AY 2014-15 K-8 
principals by location of assistant principal and teaching experience and district size 


A. Small Districts 
Assistant principal 
experience 


Same school 
Same school 


Same school 


Same school 
Same district and not the 
same school 


Same district and not the 
same school 


Same district and not the 
same school 


Same district and not the 
same school 


Only other district 


Only other district 


Only other district 
Only other district 


None 
None 


Teaching experience 


Same school 


Same district and not 
the same school 


Only other district 


None 
Same school 


Same district and not 
the same school 


Only other district 


None 


Same school 


Same district and not 
the same school 


Only other district 
None 


Same school 


Same district and not 
the same school 


Only other district 


None 


GA MA MO NC TX WA 


0.047 0.044 0.027 0.078 0.032 


0.033 0.044 0.042 0.081 0.031 


0.054 0.034 0.055 0.063 0.060 
* - 0.016 * - 


i‘ 0.018 0.046 0.029 * 


0.042 0.076 0.225 0.138 0.071 


* 0.051 0.148 0.087 0.068 
* * 0.044 * * 
* ‘ 0.007 0.008 * 
* * 0.027 0.019 * 


0.16 0.109 0.161 0.234 0.145 


* * 0.024 * * 
0.16 0.169 0.022 0.078 0.130 


0.092 0.146 0.064 0.084 0.106 


0.188 0.236 0.038 0.083 0.236 
0.167 0.039 0.051 0.014 0.068 


Notes: Individuals in the table include all principals in small school districts who work in K-8 schools (defined in 


Table 1) in the 2014-15 school year who (a) appear after the first year of each state’s panel or (b) appear in the first 
year of the panel as a teacher. For educators appearing in the first year as a teacher, all prior experience is assumed 
to be teacher experience. Prior experience includes any experience as an assistant principal or teacher in the data 
regardless of sector (e.g. K-8 or high school), and we include prior experience as an assistant principal or teacher in 
a charter school where data is available. In Texas, cells that represent less than 5 individuals are masked with an 
asterisk (*). In Georgia, Massachusetts, Missouri, and Washington, cells that represent less than 10 individuals are 


masked with an asterisk (*). 
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B. Large Districts 


Assistant Teaching GA MA MO NC TX WA 
principal experience 
experience 
Same school Same school ™ 0.051 0.018 0.033 0.020 
Same school Same district and 
not the same = 0.051 0.048 0.113 0.047 
school 
Same school Only other district * 0.054 0.028 0.034 0.024 
Same school None * * 0.015 . 
Same district and Same school 
not the same school 7 ‘ 0.026 0.020 0.015 
Same district and Same district and 
not the same school not the same 0.143 0.264 0.396 0.462 0.201 
school 
Same district and Only other district 
not the same school * 0.064 0.137 0.117 0.096 
Same district and None 
not the same school 7 x 0.084 0.010 0.022 
Only other district Same school * 0.002 x x 
Only other district Same district and 
not the same * 0.032 0.026 0.011 0.019 
school 
Only other district None * - 0.024 0.005 x 
None Same school 0.126 * 0.002 0.006 0.024 
None Same district and 
not the same 0.286 80.156 0.035 0.045 0.166 
school 
None Only other district * 0.089 0.017. 0.016 0.140 
None None 0.185 0.086 0.061 0.012 0.075 


Notes: Individuals in the table include all principals in large school districts who work in K-8 schools (defined in 
Table 1) in the 2014-15 school year who (a) appear after the first year of each state’s panel or (b) appear in the first 
year of the panel as a teacher. For educators appearing in the first year as a teacher, all prior experience is assumed 
to be teacher experience. Prior experience includes any experience as an assistant principal or teacher in the data 
regardless of sector (e.g. K-8 or high school), and we include prior experience as an assistant principal or teacher in 
a charter school where data is available. In Texas, cells that represent less than 5 individuals are masked with an 
asterisk (*). In Georgia, Massachusetts, Missouri, and Washington, cells that represent less than 10 individuals are 
masked with an asterisk (*). 
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C. All districts 


Panel C. All Districts 


Assistant Teaching GA MA MO NC TX WA 
principal experience 
experience 
Same school Same school 0.061 0.042 0.047 0.021 0.047 0.026 
Same school Same district and 
not the same 0.098 0.031 0.047 0.046 0.103 0.040 
school 
Same school Only other 
district i 0.048 0.041 0.036 0.043 0.041 
Same school None * . . 0.016 * ie 
Same districtand Same school 
not the same 0.022 . 0.017 0.032 0.023 0.014 
school 
Same district and Same district and 
not the same not the same 0.262 0.064 0.143 0.344 0.359 0.140 
school school 
Same district and = Only other 
not the same district 0.033 es 0.056 0.141 0.107 0.083 
school 
Same districtand None 
not the same . * 7 0.072 0.007 0.014 
school 
Only other district | Same school * * * 0.004 * * 
Only other district Same district and 
not the same * * 0.017 0.027 0.013 0.015 
school 
Daly OP ciswice: “Oly oret 0.041 0.136 0.105 0.106 0.153 0.140 
district 
Only other district None * 0.018 * 0.024 0.005 0.012 
None Same school 0.074 0.153 0.116 0.008 0.029 0.073 
None Same district and 
not the same 0.180 0.134 0.150 0.044 0.057 0.138 
school 
None in orber 0.055 0.162 0.184 0.023 0.037 0.185 
istrict 
None None 0.127 0.171 0.056 0.058 0.013 0.072 
Notes: Individuals in the table include all principals who work in K-8 schools (defined in Table 1) in the 2014-15 


school year who (a) appear after the first year of each state’s panel or (b) appear in the first year of the panel as a 
teacher. For educators appearing in the first year as a teacher, all prior experience is assumed to be teacher 
experience. Prior experience includes any experience as an assistant principal or teacher in the data regardless of 
sector (e.g. K-8 or high school), and we include prior experience as an assistant principal or teacher in a charter 
school where data is available. In Texas, cells that represent less than 5 individuals are masked with an asterisk (*). 
In Georgia, Massachusetts, Missouri, and Washington cells that represent less than 10 individuals are masked with 
an asterisk (*). 
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